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Mission 

•  “To  support  the  material  acquisition  process  established  by  MCO 
P5000.22  by  managing  the  Marine  Corps  Operational  Test  (OT) 
Program  for  Acquisition  Categories  (ACAT)  I  through  ACAT  IV, 
less  the  OT  of  manned  aircraft,  and  to  perform  such  other 
functions  as  directed  by  the  CMC.” 


Workforce 

•  20  of  27  Marines,  11  of  24  Civil  Service,  9  Contractors 


Scope 

•  At  least  125  programs  in  varying  stages  of  test 

•  Great  majority  non-oversight  ACAT  III/IV  programs 


AAAV 

ACAT  ID  $4  B 


MTVR 

ACAT  II  $1.4  B 


Predator 

ACAT  III  $1.9M 


JSCS 

ACAT  IVSIOK 


Across  all 
ACATs 


High  Interest  Programs 


Advanced  Amphibious  Assault  Vehicle  (AAAV) 
Lightweight  155  Howitzer  (LW-155) 

Internally  Transportable  Vehicle  (ITV) 


Maritime  Prepositioning  Force,  Enhanced,  (MPF(E)) 
USNS  GySgt  Fred  W.  Stockham 

Navy,  Marine  Corps  Intranet  (NMCI) 

LPD-17  Amphibious  Transport  Dock 


OTA  PARTNERS 


ATEC 

MGEN  J.  MARCELLO 
Auth:  1385 


DOT&E 

The  Honorable 
MR.  T.  CHRISTIE 


MCOTEA 

COL  J.  GARVIN 

Auth:  50 


JITC 

COLB.OSLER 

Auth:  est  250 


OPTEVFOR 

RADM  R.  BESAL 
Auth:  345 


AEOTEC 


MGEN  W.  PECK 
Auth:  901 


What  is  the  Commercial  IW  Threat? 


•  40%  Internal,  40%  Dial  UP  and  20%  Internet 

-  Hackers,  Crackers,  Hacktivist,  Terrorist  and  Corporate 
Espionage 

#  "Russian  Mafia"  Interactive  Week,  July  16,  2001 

-  Operates  in  50  Countries:  Infiitrate  businesses  and 
launch  internet  attacks 

•  Ministry  of  Internai  Affairs  estimates  that  5,600 
criminai  groups  (more  than  100,000  individuais)  are 
invoived  in  money  iaundering,  drugs,  and  extortion 

~  Eastern  Europeans  Crackers  among  the  most  skiiiful  in 
the  worid 

•  Led  by  former  KGB  Agents:  Some  even  plant 
employees  inside  targeted  companies 

-  Few  cases  are  prosecuted  and  thus  few  deterrents  to 
foreign  hackers! 
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Network  Centric  Warfare  relies  on  Effective 
Information  Operations 
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The  Emerging  Challenge:  Information  Assurance 


•  Effective  conduct  of  10  for  NOW  requires  that  combat  forces 
be  reliabiy  “connected”  to  the  supporting  infrastructure 

•  Information  Assurance  is  a  subset  of  10: 

~  10  that  protect  and  defend  information  and  information 
systems  (IS)  by  ensuring  their  availability,  integrity, 
confidentiality,  authentication,  non-reoudiation.  This 
includes  providing  for  the  restoration  of  IS  by 
incorporating  protection,  detection  and  reaction 
capabilities 

•  NOW  relies  on  distributed  platforms  and  sensors  to  detect, 
locate,  target  and  eliminate  enemy  with  precision  munitions 

-  Infiltrating  the  network  could  allow  the  enemy  to  exploit 
your  sensors  and  understand  your  force  disposition 

~  Simply  disrupting  the  network  isolates  sensors  from 
weapon  systems  and  renders  your  force  impotent ! 


.  attaining  one  hundred  victories  in  one  hundred  battles  is  not  the  pinnacle  of  excellence. 
Subjugating  the  enemy's  army  without  fighting  is  the  true  pinnacle  of  excellence." 

Sun  Tzu,  The  Art  of  War 
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MCOTEA  Approach:  Leverage  the  Acquisition 
Process  for  lA 

•  Effective  implementation  of  NCW  requires  we  consider 
Security,  Interoperability  and  Information  Assurance 
collectively  as  we  work  to  acquire  systems 

•  Key  documents  drive  the  acquisition  and  testing  process: 

-  DITSCAP  DOD  8510.1 

-  CJCSI  621 2.01  B  Interoperability  and  Supportability  of 
NNSS  and  IT  Systems  (08  May  2000) 

-  DOD  CIO  GIG  lA  Policy  Memo.  No.  6-8510  (16  June  2000) 

-  DOT&E  Policy  for  OT&E  of  I A  (17  Nov  1999) 

-  DOT&E  Guidelines  on  Metrics  for  OT  of  lA  (19  Jan  2001) 

•  MCOTEA  must  integrate  DOD  and  JCS  mandates  into  a 
cohesive  OT&E  Strategy  and 

-  Coordinate  strategy  with  acquisition  and  testing 
stakeholders  and  train  the  USMC  OT&E  test  force 
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DOT&E  lA  OT  Policy 


•  Policy  for  Operational  Test  and  Evaluation  of  Information 
Assurance  (1 7  Nov  1 999) 

-  Provides  Background,  Applicability  and  Scope, 
Definitions  and  Implementation 

•  Applicability 

-  ACAT 1  Programs  and  programs  with  DOT&E  oversight 
that  have  yet  to  reach  MS  “C” 

•  Policy  describes  four  implementation  steps 

-  Step  I:  Requirements,  Threat  and  Test  Documentation 
Review 

-  Step  II:  Test  Strategy  Development 

~  Step  III:  Review  lA  DT&E  and  Computer  Security 
Certification  Results  Prior  to  Entry  into  OT&E 

~  Step  IV:  Evaluation  of  lA  Vulnerabilities  during  lOT&E 


lA  OT  Four  Step  Process 
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DOT&E  lA  Metrics  Guidelines 


•  Guidelines  on  Metrics  for  Operational  Testing  of  Information 
Assurance  (19  Jan  2001) 

-  Developed  to  complement  lA  Policy 

-  Designed  to  aid  testers  and  evaluators  who  are  not 
knowledgeable  in  lA 

-  Not  all  metrics  must  be  measured  for  every  acquisition 
program 

•  T&E  Community  has  identified  eight  potential  lA  metrics 

-  Test  Standards  are  included 

•  Risk  Assessment  identifies  required  metrics! 

-  Level  1 :  No  metrics  required 

-  Level  2:  Limited  metrics 

~  Level  3:  Moderate  metrics 

-  Level  4:  All  Metrics 
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DOT&E  lA  Metrics  Guidelines 


lA  OT  Metrics 

Description 

Effectiveness  of  security  Doiicv  in  preventinq  unauthorized  access:  aii  test  standards  met? 

Effectiveness  of  system  defense  in  depth:  aii  test  standards  met? 

Metric  2A 

Effectiveness  of  system  in  preventing  unauthorized  access  (from  both  insider  and  outsider)  acceptabie 
or  unacceptabie? 

Metric  2B 

Effectiveness  of  system  in  preventinq  unnecessary  disciosure  of  system  information:  acceptabie  or 

Metric  3A 

Abiiity  to  detect  information  deqradation/corruption/attack:  acceptabie  or  unacceptabie? 

Metric  3B 

Time  (Threshoids  set  by  the  user)  to  respond  to  information  deqradation/corruption. 

Metric  3C 

Time  (Threshoids  set  by  the  user)  to  restore  deqraded/corrupted  information. 

Metric  4A 

Abiiity  to  detect  system  deqradation/corruption/attack:  acceptabie  or  not  acceptabie? 

Metric  4B 

Time  (Threshoids  set  by  the  user)  to  respond  to  system  deqradation/corruption. 

Metric  4C 

Time  (Threshoids  set  by  the  user)  to  restore  criticai  functionaiity  into  a  deqraded/corrupted  system. 

Metric  4D 

Time  (Threshoids  set  by  the  user)  to  restore  fuii  functionaiity  into  a  deqraded/corrupted  system. 

Metric  5 

Effort  (iow,  medium,  hiqh)  to  penetrate  to  a  qiven  ievei  of  access. 

Metric  6 

Effectiveness  of  authentication? 

Metrics  by  Risk  Level 

Level  2:  Low  Risk:  Red 

Level  3:  Medium  Risk:  Red  +  '  eilo'" 

Level  4:  High  Risk:  All  Metrics 


Note:  These  metrics  are  more  fully  developed  for  inclusion  in  MCOTEA  lA  OT  SOP. 
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DITSCAP  Process 


•  DoD  Information  Technology  Security  Certification  and 
Accreditation  Process  (DITSCAP)  DoD  8510.1 

-  All  IS,  to  include  stand-alone  personal  computers, 
connected  systems,  and  networks,  must  be  accredited 

~  The  standard  DoD  Approach  for  identifying  information 
security  requirements,  providing  security  solutions,  and 
managing  information  technology  system  security 

•  USMC  Project  Officer’s  Certification  and  Accreditation 
Handbook  (Sep  2000) 

•  Four  Phase  Process 

~  Phase  1:  Definition 

~  Phase  2:  Verification 

~  Phase  3:  Validation 

-  Phase  4:  Post  Accreditation 

.^^^hange^iia^warraiT^eginnin^^ie^DjTSCA^vcl^^^ 
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Leveraging  DITSCAP  for  lA 
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Joint  Interoperability 


•  CJCSI  621 2.01  B  Interoperability  and  Supportabiiity  of 
National  Security  Systems  and  Information  Technology 
Systems  (08  May  2000) 

-  Establishes  policies  and  procedures  for  J-6 

•Interoperabiiity  requirements  certification  of  MNS, 
CRD  and  ORDs 

•Supportabiiity  certification  of  C4ISPs 

•Interoperabiiity  system  validation 

-  Detaiis  a  methodoiogy  to  deveiop  interoperabiiity  KPPs 
derived  from  a  set  of  top-ievei  lERs  based  on  the  format 
and  content  of  the  C4ISR  integrated  architecture 
products 
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Leveraging  the  J-6  Certification  and  Validation 
Process 
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Integrating  Security,  Interoperability  and  lA  into 
the  MCOTEA  Process 


Test 
Concept 
i:^lopment 


Test 
Design 
Development 


Detailed 

Test 

Planning 


Interoiierjgbilifv 


Early  involvement 
with  Program  Management 
is  mandatory! 

Security  and  Interoperability  Products 
can  be  leveraged  throughout  all  phases 
of  the  MCOTEA  Process! 


Report 
Staffing 
Lessons  Lrnd 
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Final 
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Test 

Execution 


Data 
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And  Analysis 


MITRE 


Conclusions 


•  There  are  lots  of  moving  parts! 

•  MCOTEA  strategy  is  intended  to  be  tailorable  and  non 
threatening 

-  Provides  MCOTEA  an  opportunity  to  report  to  the  MDA 
regarding  how  well  policies  are  being  implemented 

•  Failure  to  implement  these  policies  puts  the  war 
fighter  at  risk  and  could  adversely  impact  USMC 
operations  in  a  Joint  Environment 

•  Early  involvement  with  DITSCAP  and  Joint  Interoperability  is 
the  key 

-  Allows  MCOTEA  to  leverage  other  activities  and  makes 
best  use  of  limited  resources 

~  Education  and  training  is  critical! 

-  MCOTEA  is  coordinating  with  JITC,  COTF,  AFOTEC, 
ATEC,  MCCDC  and  MCSC  to  refine  this  strategy! 


Backups 
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lA  Metrics  Process 


everage  DITSCAP  Phase  I  SSAA  and  DITSCAP  Phase  II  Vulnerability  Assessment 
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DITSCAP  Four  Phase  Process 
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Simplified  MCOTEA  Process 
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Simplified  MCOTEA  Process  (Concluded) 


What  are  Commercial  Organizations  doing? 


•  Corporations  are  increasing  computer  security  budgets. 

-  Recent  Gartner  reports  computer  security  expenditures 
wiil  average  4  percent  of  annual  revenue  by  201 1 

•A  tenfold  increase  from  today 

•  It  is  not  sufficient  just  to  identify  and  seal  security  holes 

~  A  system  administrator  or  security  officer  must  stand 
watch  for  "leaks"  or  intrusions 

•  Security  intelligence  professional  services  are  being  created 

-  Assume  operational  responsibility  for  securing  a 
customer's  Web  site  or  network 

-  Internet  Security  intelligence  services  are  modeled  after 
military  intelligence-gathering  apparatus 

-  A  good  security  intelligence  service  offers  alerts  and 
recommends  how  to  address  security  incidents 
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